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Abstract 
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oner’s Dilemma game is revisited here for A-players and Network 
games in the framework of Blackwell’s approachability, stochastic ap¬ 
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1 Introduction 


It has been well known for many years that mutual cooperation is a Nash 
equilibrium outcome in a two players inhnitely repeated Prisoner’s Dilemma 
game, even though defection is the dominant strategy of the one-shot game 
(see e.g the classical book by Axelrod 0 )- 

In 1980 Smale [13] studied the two players repeated Prisoner’s Dilemma 
game under the assumption that both players have limited memory and only 
keep track of the cumulative average payoffs. In this setting, he showed that 
a very simple deterministic strategy called a good strategy, if adopted by one 
player, leads to cooperation, in the sense that the other player has interest 
to cooperate. A good strategy, as dehned by Smale, is a strategy such that 
the player cooperates unless her average payoff to date is signihcantly less 
than her opponent. Later, Benaim and Hirsch [6| considered the stochastic 
analogue of Smale’s solution. In 2005, Benaim Hofbauer and Sorin [7] using 
tools from stochastic approximation and differential inclusions showed that 
the results of Smale, Benaim and Hirsch can be reinterpreted in the framework 
of Blackwell’s approachability theory [9], and that the assumption that ’’both” 
players keep track only of the cumulative average payoff is unnecessary. 

The present paper extends these works to variant of the classical Pris¬ 
oner Dilemma game including A^-Players where the underlying structure is 
a network. It is based on K. Abhyankar’s PhD thesis [1], Blackwell’s ap¬ 
proachability [9] and the stochastic approximation approach to differential 
inclusions developed in [7|. 

Section [2] sets up the notation and reviews briefly Blackwell’s approacha¬ 
bility and some of the results in [7] . Section [3] considers A^-players prisoner 
dilemma games and Section 0] prisoner dilemma games in which players are 
located at the vertices of a symmetric graph and interact only with their 
neighbors. Smale good strategies are dehned for these games and are shown 
to be Nash equilibria. 

2 Notation and Background 

Let A and B be two hnite sets representing respectively the action sets of 
some decision maker DM (for instance a player, or a group of players) and 
the action set of Nature (for instance the player’s opponents). Let U : AxB 

be a vector valued payoff function. 
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Throughout, we let E C denote the convex hull of the payoff vectors 
E = conv{f/(a, b) : a & A,b G B}. 

At discrete times n = 1,2,..., DM and Nature choose their actions 
(a„, bn) E A X B. We assume that: 

(a) The sequence {(an,^n)}n>o is a random process dehned on some proba¬ 

bility space P) and adapted to some hltration {En} (i-e {En\ is 

an increasing family of sub-a helds of E , and for each n (a^, bn) is En 
measurable). Here En has to be understood as the history up to time 
n. 

(b) Given the history En, DM and Nature act independently: 

P((®n-|-l5 ^n+l) n) P(®n-|-1 Q-l.T'^)P(^n+l b\En)- 

Let V{A) (respectively V{B)) denote the set of probabilities over A (respec¬ 
tively, B. 

A (long term) strategy for DM is a stochastic process 0 = {©n} adapted 
to {En} taking values in V{A). We say that DM uses strategy 0 if 

0n(a) = P(an+i = a\En) (1) 


for all a E A. 

The cumulative average payoff at time n is the vector 

Un = -'^U{ak,bk)EE. ( 2 ) 

k=\ 

Strategy 0 is said to be payoff-based provided 

0n(®) = Quni^) 

for all a E A, where for each u E E, Qu(-) is a probability over A and 
u E E Qu E V{A) is measurable. In this case, the family Q = {Qu}u£E is 
identihed with DM^s strategy. 

Example 1 (M-Players games) Consider an M-players game with M > 
2. Players are denoted i = 1,..., M. Player i has a hnite action set (or pure 
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strategy set) denoted S*, and a payoff function W : x ... x i—)■ for 

some > 1. 

At each discrete time n = 1, 2,... Player i chooses an action G S* and 
receives the payoff U\sf,..., sff). 

Choose DM to be some given subset of players, say I = {1,... ,k}. Set 

A = X ... X X ... X 

and [/ : S ^ X ... X ^ Ni, with 

U{a,b) = {U\a,b),...U^{a,b)). 


o 


The Limit Set theorem 

Assume that DM has a payoff-based strategy Q. For each u E E let 

c(«) = | E U{a,b)Qu{a)iy{b) ■.neV{B)y (3) 

laeA,6eS j 

The set C{u) is the convex set containing all the average payoffs that are 
obtained when DM plays the mixed strategy Qu and Nature plays any mixed 
strategy. 

Let C G E X E he the intersection of all closed subset G G E x E for 
which the fiber {y E E : {x,y) E G} is convex and contains C{x). The 
closed-convex extension of C, denoted cb(C') is dehned as 

co(C')(x) = {y E E : {x,y) E C}. 

For convenience we extend cb(C') to a set-valued map cb(C') on also 
denoted co(C'), by setting 


co(C')(x) = 00 ( 6 *) (r(a;)). (4) 

where for all x E R'^,r(a;) G E denotes the unique point in E closest to x. 
Associated to cb(C') is the differential inclusion 

fill 

^ E E{u) := -u + -^{C){u). (5) 

at 
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A solution to ([5]) is an absolutely continuous mapping t i—)■ r]{t) verifying 
fj{t) G F{T]{t)) for almost every t G M. Given such a solution, its initial 
condition is the point ri{0). Throughout, we let Su C R^) denote the set 

of all solutions to ([S]) with initial condition u. By construction, F maps points 
to non empty compact convex sets and has a closed graph. Thus, by standard 
results on differential inclusions, Su is a nonempty subset of (^^(R, R'^) htat 
iscompact (for the topology of uniform convergence on compact intervals) and 
([S]) induces a set-valued dynamical system $ = dehned for alH G R and 
n G R^ by 

^t{u) = {r]{t) : T] G Su}. 

A set A C R^ is said to be invariant for (|5]) if for all u G A there exists i] E Su 
such that r]{M.) C A (see section 3 of [7] for other notions of invariance, more 
details and references on set valued dynamics). 

A nonempty compact set A is called an attracting set for <h provided there 
is some neighborhood U of A, called a fundamental neighborhood, with the 
property that for every e > 0 there exists > 0 such that <ht(t/) C N^{A) 
for all t > te- Here stands for the e neighborhood of A. If in addition A is 
invariant, A is called an attractor. By Proposition 3.10 in [7], every attracting 
set contains an attractor with the same fundamental neighborhood. 

The basin of attraction of an attracting set A is the set 

>V(A) = {n G R^ : u^{u) C A} 

where 

a;$(M) = p|$[t,oo[(w)- 
t>o 

We let 

L = L{{uu}) 

denote the limit set of the sequence {un} dehned by (Ej). Note that L is a 
random subset of E. 

Point p G R'^ is called attainable if for any n E N and any neighborhood 
U oi p 

P(3m > n : Um, & U) > 0. 

We let Att({n,i}) denote the set of attainable points. 

Parts {i) and (n) of the following result follow from Theorems 3.6 and 3.23 
in [7] , generalizing the limit set theorem obtained for stochastic approximation 
processes (associated to an ODE) in [3l 0] and asymptotic pseudotrajectories 
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(of an ODE) in [5]. Part {Hi) follows from [10] generalizing a result obtained 
for stochastic approximation processes (associated to an ODE) in [Ij. 

Theorem 2.1 Suppose that DM uses the payoff-based strategy Q. Then with 
probability one (regardless of Nature strategy) 

(i) L = L{{un}) is almost surely an internally chain-transitive set of 

(ii) If A is an attracting set for $ then L <Z A on the event L 0 >V(A) 7 ^ 0, 

(iii) If Att{{un}) n >V(A) jtz 0 then P{L C A) > 0. 

We refer the reader to [7] for the dehnition of ’’internally chain-transitive” 
sets since this notion will not be used here but for the fact that an internally 
chain-transitive set is compact and invariant under differential inclusions l|S|). 

Approachability 

Let d denote the Euclidean distance in A set A C E is said approachable 
if there exists a long term strategy for DM such that, regardless of Nature 
strategy, 

d{um A) —^ 0. 

Given a compact subset A G E and x G E, dehne 

Ha (a;) = {y e A : d{x,A) = d{x,y)} 

where d{x, A) = inf{(i(x, y) : y E A}. 

Record that A^”(A) = {x E E : d{x,A) < r}. We say that A is a local 
B-set for the payoff-based strategy Q (or simply a local B-set) if there exists 
r > 0 such that for all x E N^{A) \ A there exists y E nA(x) such that the 
hyperplane orthogonal to [x,y] at y separates x from C{x). That is. 


{x - y,v - y) <0 


( 6 ) 


for all V E C{x) as dehned by ([3D. If A is a local B-set for all r > 0 it is 
simply called a B-set. Blackwell [9], proved that being a B-set is a sufficient 
condition for approachability. 

Theorem 2.2 Let A G E be a local B-set for the payoff-based strategy Q. 
Then 
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(i) A contains an attractor for ^ with fundamental neighborhood U = N^{A). 

In particular, 

(a) L C A on the event LDU 7 ^ 0. 

(b) Att({Mn}) n 7 ^ 0 P({-^ C A}) > 0. 

(ii) If A is a B-set, then P(L C A) = 1. 

Proof: It is proved in [7], Corollary 5.1 that A contains an attractor for 
([5]) provided inequality ([ 6 |) holds for all v G cd(C')(a:) (rather than merely 
u G C'(x)). It then suffices to prove that (jh]) also holds for all v G cd(C')(x). 

Let Graph(C') = {{x,y) E E x E : y E C{x)}. Denote its closure by 
Graph(C), and set 


Graph^(C') = {y E E : {x,y) E Graph(C')}. 

Let D{x) be the convex hull of Graph^(G). It follows from ([H]) and compact¬ 
ness of A that {x — y,v — y) < 0 for all x E E,v E Graph 3 ,(G) and some 
y E nA(a;). Clearly, this inequality still holds for all v E D{x). We claim that 
cb(C')(x) = D{x) from which the proof of {i) follows. 

Proof of the claim: The inclusion D{x) C cb(C')(a:) follows from the defi¬ 
nitions. To prove the opposite inclusion it suffices to verify that Graph(D) is 
closed. Let — )■ x, yn ^ y with G D{xn)- 

By the Caratheodory Theorem (see e.g Theorem 11.1.8.6 in [ 8 ]), the 
convex hull of a set G C equates the set obtained by taking all convex 
combinations of A^ -|- 1 points in G. Thus, there exist 

Wn = {Wn,i, ■ ■ ■,Wn,N+i) ^ Grapli,,^ (G)^+^ 


and 


«n = (ctn,i, • ■ ■) CKn,v+i) ^ (the uffit Wdimeusioual simplex of 


such that 


N+l 

yn ^ ^ 

i=l 


By compactness, after replacing sequences by subsequences we can assume 
that an ^ cx E and Wn w. Closedness of Graph(G) ensures that 
w E Graph,j,(G)^+^. Thus y E D{x). This proves the claim. 
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Assertions (a) and {b) are now consequences of Theorem I2.1[ The lastr 
statement was proved by Blackwell [^. Note that it also follows from (z). □ 


A straightforward application of this last theorem is given by the following 
result. It will be used several times in the forthcoming sections. 

Let p G with /i 7 ^ 0. For all x E set = (p, x). 

Corollary 2.3 Suppose there exist actions ai,a2 E A and numbers a, (3 such 
that for all b E B 

jjiiU (ai, b)) < a and { 02 , b)) > ft. 


Let Q be a payoff-based strategy such that 


and 

Then 

is a B—set 


fi{u) > a =» Qu{ai) = 1 
p{u) < (3^ Qu{.a 2 ) = 1 . 

A = [uE E ■. pi{u) E [a,/5]} 


Note that there is no assumption here that a < f3. li a> (3 [a, [3] stands for 
[13, a]. 

Proof: Equation ([ 6 ]) in this context becomes 

/i('u) > a ^ piiy) < a 


piu) < (3 ^ pi{y) > ft, 

for all u E E,v E C{u). By convexity of the half spaces {p(r’) < a}, {t{v) > 
/?}, and the definition of Q this is equivalent to the condition given in the 
statement of the corollary. □ 


We conclude this section with some quantitative estimates given in the 
excellent recent survey paper by Perchet [12]. Let 

lEl = sup{||n|| :vEE}, |A| = supdlvH : n G A}. 

The hrst assertion of the next theorem follows from Corollary 1.1 in [T 2 |. It 
is slight variant of a result obtained by Blackwell [9]. The second assertion 
follows from Corollary 1.5 in [T2] . 




Theorem 2.4 Suppose DM adopts the payoff-based strategy Q and that A C 
E is a B-set for Q. Then for all rj > 0 


P(sup d{um,A) >r])< 


m>n 


2(|ig| + |A|)^ 


(ii) If furthermore A is convex, 



m>n V ^ 



3 A^-Players Prisoner’s Dilemma Game 

Consider an A^-Players game (as described in Example [I]) where each player 
has two actions: cooperate C or defect D, so that S* = {C,D}. We assume 
that the payoff functions f/*, i = 1,... N, are as follows. Let s = , s^) G 

{C, be the action prohle of the players. If Player i cooperates (i.e s* = C) 
and amongst her A^ —1 opponents, k cooperate (i.e card{j ^ i : = C} = k) 

she gets 


U\s) = v{C,k). 

If she defects and amongst her opponents, k cooperate she gets 


U\s) = v{D,k). 


For k = 0,... N — 1 the numbers v{C, k), v{D, k) satisfy the following condi¬ 
tions, usual for prisoner’s dilemmas: 

(i) Defection is the dominant action : 


v{C, k) < v{D, k) 


for all A: = 0,..., A^ — 1; 

(ii) The payoff of a defector increases with the number of cooperators : 


v{D, k) < v{D, /c -I- 1) 


for all A: = 0,..., A^ — 2; 
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(iii) Mutual cooperation is a Pareto optimal: For all s G {C, D}^ 

N 

J2u‘M<Nu‘{c,...,cy, 

i=l 

Or, equivalently, for all A: = 0,..., — 1 

kv{C, k-l) + {N- k)v{D, k) < Nv{C, N-1). 

(iv) (Occasional assumption) Mutual defection is Pareto inefficient: For all 
k = 0,..., N — 1 

kv{C, k-l) + {N- k)v{D, k) > Nv{D, 0). 

Remark 1 Condition (z) makes {D,..., D) the unique Nash equilibrium of 
the one-shot game. 

Remark 2 When N = 2 we retrieve the usual two players Prisoner’s Dilemma 
game. The Pareto conditions {Hi) and {iv) amount to say that the polygon 
with vertex set 

{(u(D,0),u(D,0)); (u(D,l),u(C,0)); {v{C,0),v{D,l)y, (u(C, 1),u(C, 1))} 
is convex. 

Example 2 (Free riding) Let / : {0,..., A^} i-G- M’*' and c > 0 be such that 

^<f{k + l)-f{k)<c. 

Let 

v{C, k) = f{k -|- 1) — c and v{D, k) = f{k). 

This can be seen as a simple model of ’’free riding”. Each player can either 
Contribute (Cooperate), or Defect from contributing, to a public good. In¬ 
dividual contribution costs c and everyone -even if a defector- benefits from 
the good and is paid f{k), when there are k contributors. 

Note that the assumption on / imply that conditions (z) — {iv) above are 
satished. The fact that mutual defection is a Nash equilibrium of the one-shot 
game is reminiscent of Hardin’s book The Tragedy of the Commons [m. 
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Recall that E = conv{17(s) : s G {C, -D}^}. For u = {ui ,..., un) G E let 

1 1 ^ 

^\u) =Ui- Uj = - Uj). 

jj^i i=l 

Let (5 be a nonnegative real number, Adapting ra, 0 and [7], we define a 
6-good strategy for Player i as a payoff-based strategy Q* (as defined in section 
|2]) for the Decision Maker, Player i, such that 

g(,(C') = iif ^*(«)>o, 

and 

Q\,{D) = l\iyi\u)<-5. 

We call such a strategy continuous whenever the map ti e-)■ is continuous. 

The following result shows that, by playing a h-good strategy, a player 
(or a group of players) makes sure that her opponents’average payoff cannot 
be much better than hers, nor than the Pareto optimal payoff. Under the 
supplementary condition (iv) she ensures that her payoff cannot be much 
worse that the payoff resulting from mutual defection. If furthermore, all the 
players play a 5-good strategy, one of them being continuous, the outcome 
is the one given by mutual cooperation. As a consequence (Corollary I3.2p . 
continuous 5-good strategies form a Nash equilibrium. The proof is postponed 
to the end of the section. 

Theorem 3.1 Let k < N. Suppose that for all i G {1,..., A;} Player i plays 
a 6-good strategy. Let 

1 ^ 
j=k+l 

be the average payoff to players k 1,..., N . Then 
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(i) For all i E {I,..., k}: 


0 < lira inf < limsupw^^ — Ki ^ Tt - 

n—^OO ^ TV rn 

lira sup < v{C, N — 1), 

n—>-oo 

and, if mutual defection is inefficient, 
lira inf u\ > v{D, 0) — S. 

n^oo 


(ii) Suppose k = N. Then: 


(a) 

L{{un}) C cliag(E) = {uE E : m = ... = Un}, 

and if at least one of the players uses a continuous 6-good strategy, 
then 

(b) 

lim Un = U{C,...,C) = {v{C,N -l),...v{C,N -1)). 

n^oo 


Let £ > 0. Let 0* be a strategy (as defined by equation ([T])) for Player i. 

The strategy profile (0^,...,©^) is called an e-Nash equilibrium if for 
every i and every alternative strategy H* for i, the payoff to i resulting from 
(0 ^,..., 0*“^, S*, 0 *+^,..., Q^) cannot be £ better than the payoff resulting 
from (0^,..., 0^). More precisely: 

For every z G {1,..., A^}, every strategy S* and every S x S-valued process 
{(sn, §„)} adapted to the filtration {En} satisfying 


and 


then 


N 

P(s„+l = s|J-0 = n©n(s'), 


P(^n+1 n,) 



P(lim sup ul^ < lim inf u\ + £) 


n^oo 


n—>-oo 


1 . 
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Here 


Un 


n 


'^n 


1 

n 


k=l 


In other words, if all players but i play the equilibrium strategy, Player i 
cannot improve his payoff by more than e if he deviates from 0h 


Corollary 3.2 Let h > 0. Suppose that for alii E {I,..., N} is a contin¬ 
uous 6-good strategy. Then ..., Q^) is a 6{N — 1)-Nash eguilibrium. 

Proof: Follows from Theorem 13.11 □ 


Proof of Theorem 13.1 

For all 5 > 0 let 

A\6) = {ueE :-5< p\u) < 0}. 


Proposition 3.3 Let f 6 {1,..., A^}. Suppose that i plays a 5-good strategy 
Qh Then A®(h) is a B-set for Qh In particular, assertion {ii) of Theorem\2fM 
and Theorem fS.fl hold. 

Proof: Suppose i = 1. Given s~^ = (s^, G let k = 

card{j >1 ■. s^ = C}. Then 

AU(C'. 5-*)) = k) - v(D, k+l))<0 

and 

s-‘)) = k) - v(C, k -l))> 0. 

By Corollary 12.31 and dehnition of this concludes the proof. □ 


Proposition 3.4 (Properties of {A*(5)}) For all 6 > 0 the sets A*(h) sat¬ 
isfy the following properties: 
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(i) For a// z e {1,..., A^} and u G A*((5) 


Ui < v{C, N - 1), 
and, if mutual defection is Pareto inefficient, 


Ui > v{D, 0) — S. 


(ii) For all k < N,u E fljeji fc} z G {1,..., A:} 


0 < 


Z^j=k+l ^3 

N-k 


Ui < 6 


N -1 
N-k 


(iii) 

Pi A*(5) = diag(i?) = {u E E : Ui = ... = u^}. 

ie{i,...,N} 

Proof: Suppose z = 1. For all z] G M let 11’^ be the orthogonal projection 

onto the hyperplan {/z^(zz) = f]}. Then 


U^{u) = u-{ 


-h x 1 


where is the vector defined by /x^(zz) = (/x^,zz). That is = (/z^)i=i,...,Ar 
with fr\ = I and = 


N-l 


for z > 1. It follows that 


U^u) = ui- 


- V ^ Ez=i ^ -1 
hT N 


+ r]- 


N 


Thus, by Pareto dominance of v{C, N — 1) 


nj'(zz) < v{C,N-l) + r] 

for all u = U{s), hence for all u E E. Let now zz G A^. Then u = Il'^{u) with 
r] = fi^{u) E [—h, 0]. Thus zzi < v{C, N — 1). Similarly, if v{D, 0) is inefficient, 
then zzi > v{D, 0) — 

To prove the second assertion set A = 
that, by definition of A*(h) 


-{N -1)5 < Nui- A- B <t) 
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for all z = 1,..., A;. Thus, by summing over all z = 1,..., fc, 


-(A^ - l)kS <{N- k)A -kB <0. 


Then 


k B 

Nu,<A + B< + B 

N — k 


NB 

N-k 


and 


Nu, > A+B-{N-l)d > 


N{B - {N -1)6) 
N-k 


The last assertion is immediate, because on A*(h) Nui < 


Proof of Theorem 13. II Assertion (z) and the beginning of (zz) follow from 
Propositions 13.31 and 13.41 It remains to prove the last assertion. Assume that 
Player 1 uses a continuous strategy. Recall that C'^(x) is defined by ([2]) with 
Q = Q^. By continuity ol x ^ Q], the map x C^{x) has a closed graph 
so that cb(C')^(x) = C^{x) for all x E E. Thus, by Theorem 12.11 (z), the set 
L = L{{un\) is invariant under the differential inclusion 

u G —u + C^{u) (7) 

and, by what precedes, is contained in diag(i?). In particular, for all u E L 
there exists rj solution to ([7]) such that 7^(0) = u and ij}{rj{t)) = 0 for all t. 
Let h{t) = r]{t) + 'f]{t). Then ^^{h{t)) = 0 and h{t) E C^{r]{t)) for almost all 
t. Let V* = U{C, ...,C) = {v{C, N-1),..., v{C, N - 1)). 

By dehnition of and C^, 

ix^{u) >0^C^{u) = conv{[/(C', : s~^ E {C, 

Now, the proof of Theorem 13.31 shows that ij}{U{C,s~^) < 0 with equality 
only if s = (C, ... ,C). Thus 

/i^(zz) > 0 ^ {u e C^{u) : fi^{v) = 0} = {u*}. 

This implies that h{t) = v* and rj{t) = e~^{u — v*) + v* for all t E M.. 
By compactness of L we must have u = v* (for otherwise {'r]{t)} would be 
unbounded). 
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4 Network Prisoner’s Dilemma Games 


Network Games 

In this section we consider a game in which players are located at the vertices 
of a graph and interact only with their neighbors. There are M players de¬ 
noted i = 1,, M. Player i has a hnite action set Eh The set V = {1,..., M} 
of vertices of the graph is eqnipped with an edge set 8 C V x V. 

We assume that the graph {V,8) is 

(a) symmetric: (z, j) E 8 ^ {j,i) G 8, 

(b) self-loop free: {i,i) ^ 8, and 

(c) irreducible: for all i,jEV there exist A: > 1 and zi,..., 4 G V such that 

4 = 1,4= j and (b, h+i) G T for / = 1,..., A: — 1. 

For each {i,j) G 8 there is a real valued map 


[/O ; E* X E^' ^ R 


representing the payoff function to Player i against Player j. 

Let Neigh(z) = {j : (4 j) G 8} and let W be its cardinal. The payoff 
function to i is the map A/* : E i—)■ R'^* dehned by 

A/*(s) = (4*^(s4s^')),,Neighp). 

Using the notation of Example [H set N = and dehne the vector 

payoff function of the game as 

U = {U\ ..., U^) : E ^ R^i X ... X R^- ~ R^. 

The state space of the game is then E = conv{t/(s), s G E} C R^. 

In addition to these data, we assume given a Markov transition matrix 
K = {Kij)ij^v adapted to {y,8). That is 

Kij > 0 , ^ Kij = 1 

j 


and 

Kij > 0 (4 j) G 8. 
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The mean payoff to Player i for the strategy prohle s is dehned as 


= ( 8 ) 

j 

Irreducibility of the graph (f/, S) ensures irreducibility of the transition matrix 
K. Therefore there is a unique invariant probability tt for K. That is, 


and for alH G 


TTi > 0,^7rj = 1 
i 

j 


Dehne the weight of edge {i,j) E S as 


u. 






(9) 


Such weights will prove to be useful for dehning (5-good strategies below. Note 
that, by invariance of tt, 


-j j 


Example 3 Suppose 


Then 


f if i e Neigh (i) 
\ 0 it j ^ Neigh(z) 



E 


jGNeigh(i) 


.,V9(s) 


Ni 



and Uij 


1 

^IjGNeighp)- 


( 10 ) 


O 
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Network Prisoner’s Dilemma Games 

We consider now a particular example of network games where each pair of 
neighboring players is engaged in two players prisoner dilemma game. We 
assume that for each i G V S* = {C, D}, and 

D) = CD,..., U^\D, C) = DC, 


where 

(i) 

CD < DD <CC < DC 

as usual for the two player prisoner’s dilemma game. 

(ii) We furthermore assume that the outcome CC is Pareto optimal and that 
the outcome DD is Pareto inneficient, in the sense that for all {i,j) G S 


ipjij P ojjj^DD < ujijCD -|- ujjiDC < (Dij P ^ji)CC', 


Remark 3 If is reversible with respect to tt (meaning that Uij = uji) as 
in Example [3l Pareto inefficiency means 

2DD <CD + DC < 2CC. 

Equivalently, the polygon with vertices 

{DD,DD), {CD, DC), {DC, CD), {CC,CC) 

is convex and hence equal to E. 

For U — {Uij)i£V,j£Ne\gh(i) G M ^ X . . . X M "* 

let 

P {tl) ^ ^ 

3 

Given 5 > 0, a 5-good strategy for Player i is a payoff-based strategy Q* such 
that 

Ql{C) = l\i iP{u)>Q, 

and 

QUD) = lif p\u)<-5. 
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The following result is similar to Theorem 13.11 It shows that if a group 
of players use h-good strategies, their payoffs cannot be much worse that the 
payoff resulting from mutual defection and that a weighted average of the 
other players payoffs cannot be much better than hers. If furthermore, all 
the players play a h-good strategy, and that of player i is continuous, then 
the payoffs of i against j and j against i both equal CC, given by mutual 
cooperation. 

As a consequence (Corollary 14.2p . continuous 5-good strategies form a 
Nash equilibrium. The proof is postponed to the end of the section. 

Theorem 4.1 Assume 1 < A; < A^. Suppose that for i G {1,..., k\, Player i 
plays a 5-good strategy. Then 

(i) 

ii) 

DD -< liminf u\ < limsupu^ < CC^ (z = 1,..., /c). 

‘ZtTi n^oo 

(hi) 

N N ^ ^ k5 

y TTjDD < liminf > < limsup > < > 7rjCC-\- — . 

j=k+l j=k+l j=k+l j=k+l 

(iv) If k = N and Player I uses a continuous 5-good strategy, then for all 
j e Neigh(/) 

lim = lim = CC 

n—>-oo n—>-oo 

Corollary 4.2 Let 5 > 0. Suppose that for all i E {I,..., A^} Q* is a contin¬ 
uous 5-good strategy. Then (Q^,..., Q^) is a Nash eguilibrium. 

Proof of Theorem 14.11 

For all 5 > 0 let 

A*(5) = {uEE : -5 < in\u) < 0}. 
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Proposition 4.3 Let i G {1,..., A^}. Assume that i plays a 5-good strategy 
Q^. Then JV{5) is a B-set for Q\ 

Proof: Fix z G P. Let s = (s^,..., s^) G x ... x be such that s® = C. 
Then 

P{U(s)) = Y.‘^ti(CCti + CD(1 - t,)) -Y,Uj4CC% + DC{1 - tj))) 

j j 

where tj = 1 ii = C and 0 otherwise. Thus 

p}{U{,s)) = {CC-CD)+Y^ Ujitj{DC-CC)+CD ^ij-DC ^ 

j j j j 

< ^ij{CC - CD) + - CC) + CD - DC uji 

j j j j 

= TTiiCC-CD + DC-CC + CD - DC) = 0. 

Suppose now that s^ = D. Then 

{DCtj + DD{1 - tj)) -Y^jiiDDtj + DD{1 - tj))) 
j j 

= Y^^ij^j^DC — DD) + 'Y^^jitj{DD — CD) > 0. 
j j 

The results then follows from Corollary 12.31 □ 


Remark 4 the proof above shows that /i®(17(s)) < 0 (respectively > 0) if 
s® = (7 (resp. D) and s^ = D (resp. C) for some j ^ i 

Proposition 4.4 (Properties of {A®(h)}) For all 6 > 0 the sets {A®(h)} 
verify the following properties: 

(i) For i G A^} and u G A®(h) set Ui = Yhj KijUij. Then 

r 

DD - <Ui< CC. 

27r, - * - 
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(ii) For all k < N and u G A'^ ((5) 


(iii) 


N 



0 < 

E i*') 

u) < kS, 





N 


N 

N 

E 

7ri)DD < 


< ( Y 

i=k-\-l 


i=k-\-l 

i=k-\-l 


n 

A\6) = 

n A'(o) 




Proof: (i). Let v G be the vector defined by Vij = l,Vji = —1 for all 
j G Neigh(i) and Vki = 0 if k i, I i or {k, 1) ^ S. Let LI’' be the projection 
onto the hyperplan {fr\u) = t]} parallel to v. That is 


U^(u) = u- 


lF{u) — 7] 

fi\v) 


V = u — 


lF{u) — f] 
27ri 


Thus, for all s G S 




+ u)jiU^\s)) + r] 
2 


+ UJji)CC + T] 
2 

+ ^ji)DD + f] 
2 


nCC + 

tt,DD + I 


( 11 ) 

( 12 ) 


where the last inequalities follow from Pareto dominance. This implies that 
for all w G i? 

DD^ < Y, <^,iTV'(u}„ < CC + ^. 

* i * i * 

Hence for all m G A® 


DD -< 

27r,- 


Y,K^jU,3<CC. 

j 
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{ii). Note that 

U)j]^Hjk ^kj^kj X] ^jkUjk UJkjUkj — ^ /i-^ (m) . 

j j k k j j 

Thus, = 0 the inequalities follow from the dehnition of A^{6) 

for the hrst one and inequalities flTTl [T^ for the second one. 

(in). Let u G f]A'^(5). Then /i*(n) < 0 for all i, but since = 0, 

/x*(n) = 0. □ 


Proof of Theorem 14.11 The proof is similar to the proof of Theorem 13.11 
Assertion (z), {ii) and {Hi) follow from Propositions 14.31 and 14.41 For {iv) we 
use the fact that if player 1 plays a continuous <5 good strategy, then the limit 
set L of {n^} is an invariant set of the differential inclusion ii G —u + C^{u) 
contained in f]A*(0). By proposition 14.31 and remark 01 for all u G f]jA*(0) 
and V G C^{u) = CC. Thus, reasoning like in the proof of Theorem 

EH invariance of L shows that for all n G L = CC. 
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